Skip to content

Build GCC 15.2.0 from source as a putup BSP#6

Open
typeless wants to merge 16 commits intomainfrom
wip/example-gcc
Open

Build GCC 15.2.0 from source as a putup BSP#6
typeless wants to merge 16 commits intomainfrom
wip/example-gcc

Conversation

@typeless
Copy link
Owner

@typeless typeless commented Feb 10, 2026

Summary

Build a complete GCC 15.2.0 cross-compiler toolchain from source using putup's
3-tree mode. Validates multi-directory builds at scale: 22 Tupfiles, 3511 commands,
8 libraries, 9 executables — all from a single putup invocation.

What builds

Component Output Size
GMP, MPFR, MPC libgmp.a, libmpfr.a, libmpc.a 2.7M, 1.5M, 220K
libiberty, libbacktrace, libdecnumber .a archives 563K, 125K, 213K
libcpp, libcody .a archives 796K, 77K
C compiler cc1 51M
C++ compiler cc1plus 54M
Drivers xgcc, xg++, cpp 3M each
Tools collect2, lto-wrapper, gcov, gcov-dump 2.8-3.1M
Assembler/linker binutils (as, ar) Tupfile stubs

Smoke tested: cc1 test.c -quiet produces x86_64 assembly with GCC: (Tup) 15.2.0 ident.

Core changes

  • Symlink-safe path resolution: canonicalize paths in builder, scheduler, and dag
    expansion so ../ components resolve correctly from physical CWD in 3-tree mode
  • Parse scoping: limit Tupfile scanning to requested target directories
  • $$ escape: $$VAR in commands passes $VAR to shell (matches tup behavior)
  • GCC dep scanner: fix regex to avoid matching directory names like build-gcc/gcc/genpreds
  • Scoped config install: putup configure copies subdir tup.config files to build tree
  • is_path_in_scope: strip trailing separators for correct scope matching

Build features demonstrated

  • 3-tree builds: Tupfiles (-C), GCC sources (-S), and build output (-B) in separate trees
  • Self-contained libraries: each has its own tup.config with ?= defaults for standalone builds
  • Cross-directory groups: subdirectory objects collected via <group> into parent link commands
  • Target parameterization: targets/@(TARGET).tup isolates x86_64-specific config, backend
    objects, and driver compilation — other targets just provide a different .tup file
  • Assembly support: CPU-specific .asm → m4 → assembler pipeline for GMP
  • Per-platform configs: Linux defaults in-tree, macOS overlays in configs/host-darwin/

Test plan

  • Full build: 3511/3511 commands, 0 failures
  • cc1 smoke test: compiles C to x86_64 assembly
  • Zero "has no members" group warnings
  • All 337 unit tests pass (2624 assertions)
  • No stale macOS host strings in Linux binaries

@typeless typeless force-pushed the wip/example-gcc branch 4 times, most recently from 95d15db to 55673fb Compare February 10, 2026 10:16
Build GCC's three prerequisite libraries using putup with a 3-tree
layout (-C/-S/-B) against a read-only GCC source tree.

Each library is self-contained with ?= defaults for standalone builds.
Per-component tup.config files use prefix-free entries — the directory
scope provides the namespace via scoped config merging.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@typeless typeless changed the title Add GCC example: build GMP, MPFR, MPC from source Add GCC example with scoped configs and assembly support Feb 12, 2026
typeless and others added 2 commits February 13, 2026 13:42
Add libiberty, libdecnumber, libbacktrace, libcpp libraries and the
full gcc/ build pipeline (config headers, ~25 generator programs,
~500 backend objects, cc1 link) to the GCC example.

The generator bootstrap chain builds genmodes first, then BUILD_RTL
support objects, then all RTL generators that produce insn-*.h and
insn-*.cc from machine description files. Pre-generated files from
gengtype and genmatch are expected in gcc/pre-generated/.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
genmatch reads match.pd and generates optimized C++ for GIMPLE and
GENERIC IR simplification. It was previously requiring users to run
./configure && make on the GCC source tree to produce its outputs.

Now genmatch is compiled and linked against libcpp (for tokenization)
and run as part of the normal build. Only gengtype outputs remain in
pre-generated/.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@typeless
Copy link
Owner Author

GCC Example Build — Handoff Note

Current State

Build reaches ~1088/1740 commands before failing. The build command:

cd /home/mural/src/pup/examples/gcc && \
  PUP_IMPLICIT_DEPS=0 /home/mural/src/pup/build/putup \
  -C . -S /home/mural/src/gcc-15.2.0 -B ../../build-gcc -j$(nproc)

You need a GCC 15.2.0 source tree at /home/mural/src/gcc-15.2.0 (adjust -S path as needed).

Next Issue to Fix

4 generators with file-output flags write relative to CWD (read-only in 3-tree mode)

These generators use flags like -O, -H, -A, -D, -L, -h, -c to write output files relative to their working directory. In 3-tree mode, CWD = source dir (read-only).

Affected rules in examples/gcc/gcc/Tupfile (lines ~408-459):

Generator Flags Outputs
genattrtab -A, -D, -L insn-attrtab.cc, insn-dfatab.cc, insn-latencytab.cc
genopinit -h, -c insn-opinit.h, insn-opinit.cc
genemit -O (x10) insn-emit-{1..10}.cc
genrecog -H, -O (x10) insn-recog.h, insn-recog-{1..10}.cc

Fix: Use the same SRCDIR=$PWD && cd $(TUP_VARIANT_OUTPUTDIR) pattern already applied to genmatch (lines 308-332). Check whether %f gives absolute or relative paths in 3-tree mode — if $(MD) already resolves to absolute source paths, a simple cd $(TUP_VARIANT_OUTPUTDIR) may suffice. Test with a single generator first.

Key Discoveries (This Build Effort)

  1. $VAR (bare dollar) passes through to shell — putup's expand() only expands $(VAR). Bare $PWD, $SRCDIR etc. pass through as shell variables.

  2. Display text ^ ... ^ must be on the SAME line as |> — if on a continuation line, it gets included in the shell command and causes /bin/sh: ^: not found.

  3. GCC generators need BOTH -DIN_GCC AND -DGENERATOR_FILE — already set in HOST_CXXFLAGS in gcc/Tuprules.tup.

  4. tm_p.h must include 3 headers: config/i386/i386-protos.h, config/linux-protos.h, and tm-preds.h (declares predicate functions like register_operand).

  5. Don't set CONFIG_HAVE_UCHAR=1 — modern Linux doesn't provide uchar. GCC's coretypes.h typedefs it when HAVE_UCHAR is NOT defined.

  6. After editing tup.config, must re-run putup configure to propagate to build dir.

Workflow

Build → read first error → diagnose → fix Tupfile/config → rebuild. Repeat until cc1 links.

Separate Issue: 3-Tree Groups Bug

Order-only groups produce "has no members" warnings in 3-tree mode. Plan exists at .claude/plans/snuggly-noodling-scroll.md. This is a putup core bug, separate from the GCC example work. The GCC example uses PUP_IMPLICIT_DEPS=0 as a workaround.

@typeless typeless changed the title Add GCC example with scoped configs and assembly support Build GCC 15.2.0 from source as a putup BSP Feb 26, 2026
typeless and others added 13 commits February 26, 2026 22:33
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add $$ → $ de-escaping in the evaluator so shell variable references
survive Tupfile expansion (e.g., for f in $(names); do ... $$f).

In 3-tree builds (-C/-S/-B), the scheduler now creates source subdirs
that exist in the config tree but not the source tree, enabling packages
whose actual source lives in a separate tree (referenced via config
variables rather than the -S tree).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Build GNU binutils 2.44 cross-assembler and cross-archiver targeting
x86_64-linux from a separate source tree via @(BINUTILS_SRC). This is
the first package to use an explicit source variable rather than the
implicit -S tree, establishing the pattern for future packages.

Single Tupfile builds BFD (38 files), libsframe (2), opcodes (4),
gas (39), and ar (11) — producing working as and ar binaries that
integrate with the existing xgcc/cc1 toolchain.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The pattern required at least one token between the compiler name and
-c, so commands like `gcc -c foo.c -o foo.o` never matched. Also add
\s as an anchor alternative so compiler wrappers like `ccache gcc` are
recognized.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…mode

The scheduler auto-created source-tree directories when a config-tree
directory had no counterpart. This papered over a user setup issue
rather than honoring the 3-tree model: config tree mirrors source tree.

Makefile.pup now creates the binutils/ directory in the source tree as
an explicit setup step.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The examples/ directory becomes the BSP root (-C tree). Infrastructure
(Tupfile.ini, Tuprules.tup, Makefile.pup, configs/, scripts/) moves up
from examples/gcc/. Binutils moves to examples/binutils/ as a peer
tarball group. GCC-internal packages stay under examples/gcc/ with a
new group Tuprules.tup providing ?= defaults for standalone use.

Key changes:
- Root Tuprules.tup uses gcc/ prefix for all GCC-tarball DIR variables
- Binutils uses $(S)/$(BINUTILS_DIR) instead of @(BINUTILS_SRC)
- Nested Tupfile.ini markers removed from busybox/ and helloworld/
- Source assembly model: one source-root/ subdir per extracted tarball

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Core fixes for symlink-safe path resolution in 3-tree mode:
canonicalize source/output paths in builder, scheduler, and dag
expansion so ../  components resolve correctly from physical CWD.
Add parse_scopes to limit Tupfile scanning to requested targets.
Add $$ escape for literal dollar signs in shell commands.

GCC example cleanup after end-to-end build validation (3511 commands,
0 failures, cc1/cc1plus/xgcc/xg++/cpp all link and run):

- Delete dead config/i386/Tupfile (targets/ file compiles same sources)
- Fix stale aarch64-apple-darwin HOST_MACHINE in cp/Tupfile
- Move driver-i386 compilation to target file, parameterize via
  TARGET_DRIVER_LINK so non-x86 targets can leave it empty
- Rewrite tup.configs for Linux host (basename, sbrk, *_unlocked, etc.)
- Add missing BACKTRACE/DECIMAL_FORMAT to aarch64-linux.config
- Rewrite binutils/tup.config for Linux, add darwin overlay
- Fix Makefile.pup setup-host-configs to handle binutils path
Downloads real GCC 15.2.0 + binutils 2.44 source tarballs and builds
the full examples/ BSP using the putup binary from build-linux. Smoke-
tests cc1 by compiling a trivial C file to assembly.

Also adds PUTUP variable to Makefile.pup so CI can override the binary
path without modifying the file.
Drop clang-analyzer-* (path-sensitive symbolic execution — by far the
most expensive family). Cherry-pick modernize-* (6 checks) and
cppcoreguidelines-* (7 checks) instead of enabling full families.
Modeled after DuckDB and Ladybird browser configs.

Add 'src/|test/' file filter to run-clang-tidy so third_party TUs are
never loaded, eliminating the need for --allow-no-checks and the LLVM
21 apt install (~10 min saved per CI run). Use system clang-tidy-18
from the ubuntu-24.04 runner instead.
- gcc-example SKILL.md: update build command from examples/gcc/ to
  examples/, remove stale PUP_IMPLICIT_DEPS workaround, update phase
  list from 7 to 14, fix Phase 6 description, fix scope description
- tupfile-patterns SKILL.md: fix root Tuprules example to use gcc/gmp
  paths matching actual BSP layout
- examples/README.md: add Makefile Targets + Variables section, fix
  Architecture wording (not all dirs have gcc/ prefix), fix Known
  Limitations (no Tupfile.ini, not $(CC))
- examples/Tupfile: fix stale reference to gcc/gcc/config/i386/Tupfile
  (now gcc/gcc/targets/x86_64-pc-linux-gnu.tup)
- examples/gcc/gcc/Tupfile: update header comment to 14 phases
- examples/busybox/README.md: remove deleted Tupfile.ini from table
- docs/reference.md: add -C/-S to configure Relevant Options
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant